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PART I: Summary of Required Elements for State Accountability 
Systems 

Instructions 

The following chart is an overview of States' implementation of the critical elements 
required for approval of their State accountability systems. States must provide detailed 
implementation information for each of these elements in Part II of this Consolidated 
State Application Accountability Workbook. 

For each of the elements listed in the following chart, States should indicate the current 
implementation status in their State using the following legend: 

F: State has a final policy, approved by all the required entities in the State (e.g., State 
Board of Education, State Legislature), for implementing this element in its 
accountability system. 

P: State has a proposed policy for implementing this element in its accountability 
system, but must still receive approval by required entities in the State (e.g., 
State Board of Education, State Legislature). 

W: State is still working on formulating a policy to implement this element in its 
accountability system. 
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Summary of Implementation Status for Required Elements of 
State Accountability Systems 



Status 



Principle 1: All Schools 



State Accountability System Element 



F 



F 



F 



F 



F 



F 



1 .1 Accountability system includes all schools and districts in the state. 

1 .2 Accountability system holds all schools to the same criteria. 

1 .3 Accountability system incorporates the academic achievement standards. 

1 .4 Accountability system provides information in a timely manner. 

1 .5 Accountability system includes report cards. 

1 .6 Accountability system includes rewards and sanctions. 



Principle 2: All Students 



F 



F 



F 



2.1 The accountability system includes all students 

2.2 The accountability system has a consistent definition of fuii academic year. 

2.3 The accountability system properly includes mobile students. 



Principle 3: Method of AYP Determinations 



3.1 Accountability system expects all student subgroups, public schools, and LEAs to reach 
proficiency by 2013-1 4. 

3.2 Accountability system has a method for determining whether student subgroups, public 
schools, and LEAs made adequate yearly progress. 

3.2a Accountability system establishes a starting point. 

3.2b Accountability system establishes statewide annual measurable objectives. 

3.2c Accountability system establishes intermediate goals. 



Principle 4: Annual Decisions 



4.1 The accountability system determines annually the progress of schools and districts. 



STATUS Legend: 

F - Final state policy 

P - Proposed policy, awaiting State approval 
W - Working to formulate policy 
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Principle 5: Subgroup Accountability 



5.1 The accountability system includes all the required student subgroups. 

5.2 The accountability system holds schools and LEAs accountable for the progress of student 
subgroups. 

5.3 The accountability system includes students with disabilities. 

5.4 The accountability system includes limited English proficient students. 

5.5 The State has determined the minimum number of students sufficient to yield statistically 
reliable information for each purpose for which disaggregated data are used. 

5.6 The State has strategies to protect the privacy of individual students in reporting 
achievement results and in determining whether schools and LEAs are making adequate 
yearly progress on the basis of disaggregated subgroups. 



Principle 6: Based on Academic Assessments 

6.1 Accountability system is based primarily on academic assessments. 



Principle 7: Additional Indicators 



7.1 Accountability system includes graduation rate for high schools. 

7.2 Accountability system includes an additional academic indicator for elementary and middle 
schools. 

7.3 Additional indicators are valid and reliable. 



Principle 8: Separate Decisions for Reading/Language Arts and Mathematics 



8.1 Accountability system holds students, schools and districts separately accountable for 
reading/language arts and mathematics. 



Principle 9: System Validity and Reliability 



9.1 Accountability system produces reliable decisions. 

9.2 Accountability system produces valid decisions. 

9.3 State has a plan for addressing changes in assessment and student population. 



Principle 10: Participation Rate 



10.1 Accountability system has a means for calculating the rate of participation in the statewide 
assessment. 

1 0.2 Accountability system has a means for applying the 95% assessment criteria to student 
subgroups and small schools. 

STATUS Legend: 

F - Final policy 

P - Proposed Policy, awaiting State approval 
W- Working to formulate policy 
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PART II: State Response and Activities for Meeting State 
Accountability System Requirements 



Instructions 

In Part II of this Workbook, States are to provide detailed information for each of the 
critical elements required for State accountability systems. States should answer the 
questions asked about each of the critical elements in the State's accountability system. 
States that do not have final approval for any of these elements or that have not 
finalized a decision on these elements by January 31 , 2003, should, when completing 
this section of the Workbook, indicate the status of each element that is not yet official 
State policy and provide the anticipated date by which the proposed policy will become 
effective. In each of these cases, States must include a timeline of steps to complete to 
ensure that such elements are in place by May 1, 2003, and implemented during the 
2002-2003 school year. By no later than May 1 , 2003, States must submit to the 
Department final information for ail sections of the Consolidated State Application 
Accountability Workbook. 
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PRINCIPLE 1. A single statewide Accountability System applied to all public 
schools and LEAs. 



CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING STATUTORY 
REQUIREMENTS 


EXAMPLES OF 
NOT MEETING 
REQUIREMENTS 


1 .1 How does the State 
Accountability System 
include every public school 
and LEA in the State? 


Every public school and LEA is 
required to make adequate 
yearly progress and is included in 
the State Accountability System. 

State has a definition of “public 
school” and “LEA” for AYP 
accountability purposes. 

• The State Accountability 
System produces AYP 
decisions for all public 
schools, including public 
schools with variant grade 
configurations (e.g. , K-1 2), 
public schools that serve 
special populations (e.g., 
alternative public schools, 
juvenile institutions, state 
public schools for the blind) 
and public charter schools. 

It also holds accountable 
public schools with no 
grades assessed (e.g., K- 
2). 


A public school or LEA is not 
required to make adequate 
yearly progress and is not 
included in the State 
Accountability System. 

State policy systematically 
excludes certain public schools 
and/or LEAs. 


STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 


North Carolina’s pre-existing School-Based Management and Accountability Program (called 
the ABCs) included every school and LEA in the state. Our implementation of AYP will also 
include every school and LEA in the state. School AYP also will serve as a “closing the 
achievement gap” component of the ABCs in compliance with N.C. General Statutes § 1 15C- 
105.35. Additionally, incentive awards will be provided at state expense (pending legislative 
approval) for schools that make AYP. The State Board of Education incorporated AYP into the 
ABCs by action at its June, 2002 meeting. 



7 





CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING STATUTORY 
REQUIREMENTS 


EXAMPLES OF 
NOT MEETING 
REQUIREMENTS 


1.2 How are all public schools 
and LEAs held to the same 
criteria when making an AYP 
determination? 


All public schools and LEAs are 
systematically judged on the 
basis of the same criteria when 
making an AYP determination. 

If applicable, the AYP definition is 
integrated into the State 
Accountability System. 


Some public schools and LEAs 
are systematically judged on the 
basis of alternate criteria when 
making an AYP determination. 



STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 



AYP will be calculated in the same manner for all schools and LEAs in accordance with the 
requirements in NCLB and the Final Regulations. Based on State Board of Education action in 
June, 2002 AYP will be incorporated into the state’s accountability system, the ABCs, as a 
“closing the achievement gap” component pursuant to North Carolina General Statute §1 15C- 
105.35. 

The State Board of Education, however, will continue to calculate the growth component of the 
ABCs Accountability Program using the alternative schools model to designate which alternative 
schools have made the equivalent of expected growth or high growth. Note that all students in 
the alternative schools take the same state tests as those in the traditional public schools. 

Evidence: Revised APA Policy HSP-C-005, and http://sbepolicv.dpi.state.nc.us/ ; look for HSP- 
C-013 
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CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING STATUTORY 
REQUIREMENTS 


EXAMPLES OF 
NOT MEETING 
REQUIREMENTS 


1 .3 Does the State have, at a 


State has defined three levels of 


Standards do not meet the 


minimum, a definition of 


student achievement: basic, 


legislated requirements. 


basic, proficient and 
advanced student 


proficient and advanced 7 




achievement levels in 


Student achievement levels of 




reading/language arts and 


proficient and advanced 




mathematics? 


determine how well students are 
mastering the materials in the 
State’s academic content 
standards; and the basic level of 
achievement provides complete 
information about the progress of 
lower-achieving students toward 
mastering the proficient and 
advanced levels. 





STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 

Yes. The state has four achievement levels defined for its End-of-Grade tests (grades 3-8) in 
reading and mathematics, the North Carolina High School Comprehensive Test of Reading and 
Mathematics (grade 10), the Algebra I End-of-Course tests, the English I End-of-Course tests, 
and the Grade 10 Writing Assessment. Levels III and IV indicate grade level proficiency or 
above. The achievement levels will be used in the calculation of AYP. 

Historically, equating or linking studies are carried out whenever there is a transition in the 
curriculum and related tests to assure that the achievement level standards remain comparable 
across editions of assessments. However, on October 12, 2006, the State Board of Education 
approved higher standards for the third edition of the mathematics assessments in grades 3-8 
with an accompanying recalculation of the annual measurable objectives as reflected on page 23 
of the workbook. 

Evidence: http://www.ncpublicschools.org/docs/accountability/testing/eog/asb_achlev.pdf and 
http://www.ncpublicschools.org/newsroom/news/2006-07/20061012-01 



1 System of State achievement standards will be reviewed by the Standards and Assessments Peer 
Review. The Accountability Peer Review will determine that achievement levels are used in determining 
AYP. 
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CRITICAL ELEMENT 



1.4 



How does the State provide 
accountability and adequate 
yearly progress decisions 
and information in a timely 
manner? 



EXAMPLES FOR 
MEETING STATUTORY 
REQUIREMENTS 



State provides decisions about 
adequate yearly progress in time 
for LEAs to implement the 
required provisions before the 
beginning of the next academic 
year. 



EXAMPLES OF 
NOT MEETING 
REQUIREMENTS 



Timeline does not provide 
sufficient time for LEAs to fulfill 
their responsibilities before the 
beginning of the next academic 
year. 



State allows enough time to 
notify parents about public 
school choice or supplemental 
educational service options, 
time for parents to make an 
informed decision, and time to 
implement public school 
choice and supplemental 
educational services. 



STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 



In North Carolina, tests are scored locally within each LEA. The Department of Public 
Instruction provides software for scoring as well as for analysis. Existing software is being 
modified so that AYP calculations can be completed locally. 

NCDPI anticipates releasing software to the LEAs in June 2003 to enable them to generate AYP 
results for their respective schools. LEAs will be instructed to publicly announce AYP results 
for their schools before June 30 2003 (prior to the beginning of the next school year). 

In addition, all Title I schools that did not make AYP during the 2001-02 school year (total of 48 
schools) were asked to send representatives to NCDPI for a meeting with Dr. Elsie Leak, 
Associate State Superintendent for Curriculum and School Reform, Mr. Bill McGrady, Section 
Chief for Compensatory Education and Mr. Gongshu Zhang, Title I Evaluation Consultant, to 
review their results from last year and discuss the steps needed to invoke NCLB sanctions if their 
school does not make AYP again this year. 

North Carolina will report adequate yearly progress (AYP) results for the assessments in the 
2005-06 school year under a staggered schedule. Due to the implementation of new mathematics 
assessments and subsequent standards setting, the State will not report elementary and middle 
school AYP mathematics results until October. Consequently, AYP determinations for 
elementary and middle schools and local educational agencies (LEAs) based upon 
reading/language arts results will be made in July. The State will prioritize AYP mathematics 
calculations for those schools that are on the State “watch list” (i.e. those schools and districts 
that have failed to make AYP for at least one year). Schools and districts that are identified for 
improvement based upon the 2005-06 mathematics results will begin offering interventions 
immediately based upon the data released in October. 
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CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING STATUTORY 
REQUIREMENTS 


EXAMPLES OF 
NOT MEETING 
REQUIREMENTS 


1 .5 Does the State Accountability 
System produce an annual 
State Report Card? 


The State Report Card includes 
all the required data elements 
[see Appendix A for the list of 
required data elements]. 

The State Report Card is 
available to the public at the 
beginning of the academic year. 


The State Report Card does not 
include all the required data 
elements. 

The State Report Card is not 
available to the public. 




The State Report Card is 
accessible in languages of major 
populations in the State, to the 
extent possible. 






Assessment results and other 
academic indicators (including 
graduation rates) are reported by 
student subgroups 





STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 



Yes. It has been designed to accommodate the elements in Appendix A as well as a wealth of other 
information. 

The disaggregations of state data are generally completed around October, which is typically within two 
months after the ABCs results are released. The 2002 NC Report Card is currently on schedule for 
release in February 2003. There exists a Report Card Committee that will continue working to find ways to 
release the report card earlier. 

The North Carolina School Report Cards for 2002, including schools, LEAs and State are on the DPI 
website at: http://www.ncreportcards.org/src/ . These reports are supported by additional grade level and 
disaggregation detail (e.g., two year trends in percent proficient in each subject and grade level), found at 
http://www.ncpublicschools.ora/vol2/rsds2002/index.html , and by additional reporting in the First in 
America report published by the NC Education Research Council (NCERC) at 
http://erc.northcarolina.edu/content.php/svstem/fia.htm . 

These reports do not currently address all of the specific requirements in NCLB because: 1) they were 
designed before DPI had complete details about the requirements; and 2) because the first year of 
implementation for NCLB has not yet concluded and thus all of the necessary data are not yet available. 
However, the NCDPI, the NCERC and the NC Governor’s office have coordinated to produce a 
comprehensive reporting system that will accommodate NCLB requirements. 

The 2002 release of School Report Cards demonstrates the flexibility of the report card design to 
accommodate NCLB requirements. The supplemental disaggregated reports at the second website reflect 
the framework of the underlying database to support the disaggregation required by NCLB. The school 
report card committee that produced the 2002 report is working now to incorporate the NCLB reporting 
requirements, as specified in the law, the regulations and the Accountability Workbook (Appendix A) 
submitted January 31 , 2003. 
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CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING STATUTORY 
REQUIREMENTS 


EXAMPLES OF 
NOT MEETING 
REQUIREMENTS 


1 .6 How does the State 


State uses one or more types of 


State does not implement 


Accountability System 


rewards and sanctions, where the 


rewards or sanctions for public 


include rewards and 


criteria are: 


schools and LEAs based on 


sanctions for public schools 
and LEAs? 2 


• Set by the State; 

• Based on adequate yearly 
progress decisions; and, 

Applied uniformly across public 
schools and LEAs. 


adequate yearly progress. 



2 The state must provide rewards and sanctions for all public schools and LEAs for making adequate 
yearly progress, except that the State is not required to hold schools and LEAs not receiving Title I funds 
to the requirements of section 1 1 1 6 of NCLB [§200.1 2(b)(40)]. 
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STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 



The State Board of Education revamped the incentive awards structure in June 2002 to include AYP as a 
basis for awards in addition to the usual ABCs criteria. Under the approved structure certified staff in 
public schools will receive up to $600 each if their school makes expected growth in the ABCs, up to an 
additional $600 if the school makes high growth, and $600 each if the school makes AYP for a maximum 
possible award of $1800 per certified staff if the schools makes all three goals. Teacher assistants can 
receive up to $200 each for each component, up to a maximum of $600. 

SBE again will seek legislative action in the spring of 2004 to allow budgetary allotments to support the 
proposed incentive awards. 

Communication will play a major role in how the public views the accountability results from this school 
year. The NCDPI recently has been conducting “Media Briefings” across the state so that members of the 
media can have an opportunity to better understand NCLB . Paid of the conversation involves the fact that 
the SBE will be making changes to the recognition categories to incorporate AYP stalling with the 2003- 
2004 school year. The SBE (based on input from various advisory committees) wants to see data from the 
first year of NCLB implementation to better guide the final decisions about the changes in recognition 
categories. The SBE, whenever possible, avoids making major changes in the accountability program in 
the middle of the school year. Therefore, the public will be informed that this year’s categories reflect the 
school as a whole while the AYP results reflect subgroup performance as well. 

The following proposed timeline is presented for SBE changes to the recognition categories (it will be 
done sooner if data analyses are completed earlier than expected). 

> June/July 2003 - Local Education Agencies (LEAs) report AYP results for their respective schools 
(prior to the beginning of the next school year); LEAs submit data to NCDPI for additional quality control 
and to do ABCs analyses and verify LEA AYP decisions 

> September 2003 - SBE approves ABCs results and verifies AYP results reported by the LEAs). 

> October 2003 - Compliance Commission for Accountability and other advisory committees review 
options for changes in recognition categories. 

> November 2003 - SBE discusses recommendations from various committees. 

> January 2004 — SBE approves the addition of a new ABCs recognition category, “Schools of 
Excellence Plus” that will signify Schools of Excellence that make AYP, effective with the 2003-04 
school year. In March 2004 the SBE officially names this new top recognition category Honor Schools of 
Excellence. 

The State will not hold non-Title I schools to the requirements of section 1 1 16 of NCLB [§200. 12(b)(40)]; 
non-Title I schools that do not make AYP for two consecutive years due to missing one or more targets in 
the same subject area, (e.g., reading) will be required to amend their school improvement plan to address 
the situation. 
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PRINCIPLE 2. All students are included in the State Accountability System. 



CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING STATUTORY 
REQUIREMENTS 


EXAMPLES OF 
NOT MEETING 
REQUIREMENTS 


2.1 How does the State 


All students in the State are 


Public school students exist in 


Accountability System 


included in the State 


the State for whom the State 


include all students in the 


Accountability System. 


Accountability System makes 


State? 


The definitions of “public school” 
and “LEA" account for all 
students enrolled in the public 
school district, regardless of 
program or type of public school. 


no provision. 



STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 



Every public school in the state is assigned a six-digit LEA/SCHOOL code. Each school 
maintains a student information management system (SIMS) for purposes of recording 
membership, attendance, grades and other information about each student. When statewide 
tests are administered, schools are required to submit an answer sheet for every student in 
membership. Data collected for accountability purposes are submitted to numerous quality 
control checks to ensure that they are complete, accurate and consistent with the SIMS records. 
Operational procedures are spelled out in a variety of documents distributed to LEA Testing 
Coordinators who are trained through a system of Regional Accountability Coordinators who 
assist local personnel in maintaining and reporting their data according to agency requirements. 

A full spectrum of assessment alternatives is offered ranging from multiple-choice tests to 
performance assessments and alternative assessments to allow every student to be tested in an 
appropriate manner. 

Students who are absent may impact schools’ ABCs outcomes by causing them to fail to meet 
testing requirements to test at least 95% of all students. The 95% requirement for NCLB will 
also be administered as part of AYP starting in 2002-03. Beginning in 2003-04, North Carolina 
will average participation rates for the last two or three years, depending upon how many years 
of data are available, whenever a school fails to meet the 95% tested standard for the current 
year. Schools that meet the 95% tested standard using the average will be noted in the reports. 
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CRITICAL ELEMENT 



EXAMPLES FOR 
MEETING STATUTORY 
REQUIREMENTS 



EXAMPLES OF 
NOT MEETING 
REQUIREMENTS 



2.2 How does the State define The State has a definition of “full LEAs have varying definitions of 

“full academic year” for academic year” for determining “full academic year.” 

identifying students in AYP which students are to be included 

decisions? in decisions about AYP. The State’s definition excludes 

students who must transfer from 

The definition of full academic one district to another as they 

year is consistent and applied advance to the next grade, 

statewide. 

The definition of full academic 
year is not applied consistently. 



STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 

By action in November 2002, the SBE defines FAY as 140 days in membership as of the first 
day of EOG testing (which occurs during the final three weeks of school.) For Grade 10, FAY is 
defined as 140 days in membership as of the first day of spring EOC testing starting with the 
2004-05 school year. 



Evidence: http://www.ncpublicschools.org/sbehighlights/nov02highlights.html 
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CRITICAL ELEMENT 



2.3 



EXAMPLES FOR 
MEETING STATUTORY 
REQUIREMENTS 



EXAMPLES OF 
NOT MEETING 
REQUIREMENTS 



How does the State 
Accountability System 
determine which students 
have attended the same 
public school and/or LEA for 
a full academic year? 



State holds public schools 
accountable for students who 
were enrolled at the same public 
school for a full academic year. 

State holds LEAs accountable for 
students who transfer during the 
full academic year from one 
public school within the district to 
another public school within the 
district. 



State definition requires students 
to attend the same public school 
for more than a full academic 
year to be included in public 
school accountability. 

State definition requires students 
to attend school in the same 
district for more than a full 
academic year to be included in 
district accountability. 



State holds public schools 
accountable for students who 
have not attended the same 
public school for a full academic 
year. 



STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 



Membership information from SIMS is contained in the master accountability files collected 
from each school. These files indicate the number of days in membership in the current school. 
Unfortunately the SIMS does not contain information about membership at previously attended 
schools whenever students transfer from one school to another. North Carolina plans to add a 
process for the Spring 2003 data collection that will require schools to code in the master 
accountability files whether the students who have not satisfied FAY in the current school have 
been in other schools in the LEA during the current year for a total of 140 days. 

All students will be counted in the calculation of state AYP irrespective of the amount of time 
that they have attended schools in the state during the current year since there is no reliable 
method currently available to track the information statewide. The state is implementing a new 
student information system called NCWISE that can handle these data but it will be several years 
before it is installed in all schools. 
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PRINCIPLE 3. State definition of AYP is based on expectations for growth in 
student achievement that is continuous and substantial, such that all students 
are proficient in reading/language arts and mathematics no later than 2013-2014. 



CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING REQUIREMENTS 


EXAMPLES OF 
NOT MEETING 
REQUIREMENTS 


3.1 How does the State’s 

definition of adequate yearly 
progress require all students 
to be proficient in 
reading/language arts and 
mathematics by the 201 3- 
2014 academic year? 


The State has a timeline for 
ensuring that all students will 
meet or exceed the State’s 
proficient level of academic 
achievement in reading/language 
arts 3 and mathematics, not later 
than 2013-2014. 


State definition does not require 
all students to achieve 
proficiency by 2013-2014. 

State extends the timeline past 
the 2013-2014 academic year. 



STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 



The state’s AYP timeline requires that all students reach proficiency no later than 2013-2014. 

See 3.2c. 

As approved by the US Department of Education through a flexibility agreement, North Carolina 
will implement a growth model as a means of determining AYP for schools and districts. For 
details regarding the proposal, including the approval letter, refer to: 

www.ed.gov/admins/lead/account/growthmodel/nc/index.html . It is important to note that the 
growth model is not impacted by a change in proficiency standards due to the structure of the 
calculation, the new more rigorous standards will be used as the targets. 



3 If the state has separate assessments to cover its language arts standards (e.g., reading and writing), 
the State must create a method to include scores from all the relevant assessments. 
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CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING REQUIREMENTS 


EXAMPLES OF 
NOT MEETING 
REQUIREMENTS 


3.2 How does the State 
Accountability System 
determine whether each 
student subgroup, public 
school and LEA makes 
AYP? 


For a public school and LEA to 
make adequate yearly progress, 
each student subgroup must 
meet or exceed the State annual 
measurable objectives, each 
student subgroup must have at 
least a 95% participation rate in 
the statewide assessments, and 
the school must meet the State’s 
requirement for other academic 
indicators. 

However, if in any particular year 
the student subgroup does not 
meet those annual measurable 
objectives, the public school or 
LEA may be considered to have 
made AYP, if the percentage of 
students in that group who did 
not meet or exceed the proficient 
level of academic achievement 
on the State assessments for that 
year decreased by 1 0% of that 
percentage from the preceding 
public school year; that group 
made progress on one or more of 
the State’s academic indicators; 
and that group had at least 95% 
participation rate on the 
statewide assessment. 


State uses different method for 
calculating how public schools 
and LEAs make AYP. 
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STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 



For a public school and LEA to make adequate yearly progress, each student subgroup must meet or exceed the 
State's annual measurable objectives, each student subgroup must have at least a 95% participation rate (North 
Carolina will average participation rates for the last two or three years, depending on how many years of data are 
available, whenever a school does not meet the 95% standard for the current year) in the statewide assessments, 
and the school must meet the State’s requirement for graduation rate or attendance. (Graduation rate is the other 
academic indicator in schools that have a twelfth grade and graduate seniors. Attendance is the other academic 
indicator in schools that do not have a twelfth grade and/or do not graduate seniors.) For LEA and state AYP both 
attendance and graduation rate will be used as the Other Academic Indicators (OAI). 

To determine whether annual measurable objectives for AYP are met, a 95% confidence interval will be used 
around the percentages of students scoring proficient in reading and/or mathematics to determine whether the 
standards have been met, effective with the 2003-04 school year. For schools that meet an annual measurable 
objective through application of the confidence interval, actual proficiency percentages will be reported with a note 
indicating that the confidence interval was applied. 

However, if in any particular year a student subgroup does not meet those annual measurable objectives, the 
public school or LEA will be considered to have made AYP, if the percentage of students in that group who did not 
meet or exceed the proficient level of academic achievement on the State assessments for that year decreased by 
10% of that percentage from the preceding public school year; that group made progress on the applicable 
academic indicator; and that group had at least 95% participation rate on the statewide assessment. (We refer to 
this as the “safe harbor” provision.) (For the 2005-06 school year for Mathematics, the basis for Safe Harbor will be 
the percentage of students in the school in the subgroup in 2004-05 who would have been proficient using the new 
standard implemented in the 2005-06 school year. This will be determined using the concordance table from the 
linking study to determine at what point on the old scale, the new standard would have been placed.) 

A Title I school not making AYP must have missed one or more targets in the same subject (e.g., reading) for two 
successive years before it would be required to enter Title I School Improvement. 

Targeted Assistance Options: 

Title I Targeted Assistance Schools (TAS) are given options to determine AYP differently from other schools. They 
can select how to determine AYP only using students actually served in the TAS program or eligible to be served in 
the TAS program. To select these options there are several procedures that must be followed per the February 4, 
2004 letter to Dr. Mike Ward from Raymond Simon from the USED. However, more recent information was 
provided April 29, 2004 from the USED that warrants additional changes. 

1 . Initially, the NCDPI will analyze AYP results for TAS using all students in the school. If the TAS does not 
meet AYP, then the results for Title I served students will be analyzed. If the TAS again does not meet 
AYP, then the results for students eligible for Title I will be analyzed. These additional analyses are 
contingent upon the LEA “tagging” the appropriate students for the analyses. If the students are not 
“tagged,” then that is the LEA's way of informing us that they are not interested in us conducting the 
additional analyses. 

2. Unless the TAS meets the minimum number of 40 students being served or eligible to be served then the 
results will be based on all students in the school. 

3. If the school provides services in only one subject area then the LEA must determine if the results for AYP 
for each subject will be based on only the served students, only the eligible students or all of the students 
in the school. 

4. The other academic indicator can be calculated on any of the options unless safe harbor needs to be 
invoked. In that case the same criteria must be used for comparison purposes (only those served or only 
those eligible). If this is not possible then the other academic indicator must be based on all students in 
the school. 

5. Results from all students in the TAS will be used for making AYP decisions at the LEA and State levels. 

6. For schools without tested grades the AYP can be determined by back-mapping or forward mapping for 
students. 
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CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING REQUIREMENTS 


EXAMPLES OF 
NOT MEETING 
REQUIREMENTS 


3.2a What is the State’s starting 
point for calculating 
Adequate Yearly 
Progress? 


Using data from the 2001-2002 
school year, the State 

established separate starting 
points in reading/language arts 
and mathematics for measuring 
the percentage of students 
meeting or exceeding the State’s 
proficient level of academic 
achievement. 

Each starting point is based, at a 
minimum, on the higher of the 
following percentages of students 
at the proficient level: (1 ) the 
percentage in the State of 
proficient students in the lowest- 
achieving student subgroup; or, 
(2) the percentage of proficient 
students in a public school at the 
20 th percentile of the State’s total 
enrollment among all schools 
ranked by the percentage of 
students at the proficient level. 

A State may use these 
procedures to establish separate 
starting points by grade span; 
however, the starting point must 
be the same for all like schools 
(e.g., one same starting point for 
all elementary schools, one same 
starting point for all middle 
schools...). 


The State Accountability System 
uses a different method for 
calculating the starting point (or 
baseline data). 



20 





STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 



Using the data from the 2001-02, 2000-01 and the 1999-2000 school years for EOG (grades 3-8), 
and averaging the results for the three years, the State established separate starting points in 
reading/language arts and mathematics for measuring the percentage of students meeting or 
exceeding the State’s proficient level of academic achievement in grades 3-8. Using the data 
from the 2000-01, 1999-2000 and 1998-99 school years for the NC High School Comprehensive 
Test of Reading and Mathematics (grade 10), and averaging the results for the three years, the 
State established separate starting points in reading/language arts and mathematics for measuring 
the percentage of students meeting or exceeding the State’s proficient level of academic 
achievement in grade 10. Beginning in the 2004-05 school year, data from the 2003-04 school 
year for the Algebra I EOC tests, the English I EOC tests, and the Grade 10 Writing Assessments 
along with alternate assessments were used to set starting points for grade 10. (Note - the North 
Carolina High School Comprehensive Test was administered in 2001-2002 only in the high 
schools serving Title I students in grades 10-12.) 

Each starting point calculation is based on the higher of the following percentages of students at 
the proficient level: (1) the percentage in the State of proficient students in the lowest-achieving 
student subgroup; or, (2) the percentage of proficient students in a public school at the 20 th 
percentile of the State’s total enrollment among all schools ranked by the percentage of students 
at the proficient level. 

The starting points so established are the same for all like schools (e.g., schools containing any 
of the grades 3-8 or schools containing grade 10.) Schools containing grades from both grade 
ranges will be required to meet the intermediate goals and annual measurable objectives for both 
grade ranges. 



The starting points are as follows for n=40: 



Grade Ranges 


Reading/Language Arts 


Mathematics 


3-8 


68.896550615986 
(Rounded to 68.9) 


74.6495245710527 
(Rounded 74.6) 








10 


52.0114667940755 
(Rounded to 52.0) 


54.8777467227979 
(Rounded to 54.9) 








Beginning in 2004-05 
10 


35.4304635 
(Rounded to 35.4) 


70.8333333 
(Rounded to 70.8) 















Note: See page 24 for intermediate goals. 
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CRITICAL ELEMENT 



EXAMPLES FOR EXAMPLES OF 

MEETING REQUIREMENTS NOT MEETING 

REQUIREMENTS 



3.2b What are the State’s annual 
measurable objectives for 
determining adequate yearly 
progress? 



State has annual measurable 
objectives that are consistent 
with a state’s intermediate goals 
and that identify for each year a 
minimum percentage of students 
who must meet or exceed the 
proficient level of academic 
achievement on the State’s 
academic assessments. 

The State’s annual measurable 
objectives ensure that all 
students meet or exceed the 
State’s proficient level of 
academic achievement within the 
timeline. 



The State Accountability System 
uses another method for 
calculating annual measurable 
objectives. 

The State Accountability System 
does not include annual 
measurable objectives. 



The State’s annual measurable 
objectives are the same 
throughout the State for each 
public school, each LEA, and 
each subgroup of students. 



STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 



□ Annual measurable objectives are set consistent with the starting points and intermediate 
goals established by the SBE. See 3.2c. 
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CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING REQUIREMENTS 


EXAMPLES OF 

NOT MEETING REQUIREMENTS 


3.2c What are the State’s 


State has established intermediate 


The State uses another method for 


intermediate goals for 
determining adequate 


goals that increase in equal 
increments over the period covered by 


calculating intermediate goals. 


yearly progress? 


the State timeline. 

•The first incremental increase 
takes effect not later than the 
2004-2005 academic year. 

• Each following incremental 
increase occurs within three 
years. 


The State does not include 
intermediate goals in its definition of 
adequate yearly progress. 



STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 



The SBE established in January 2003 that intermediate goals shall change in three-year increments. The 
first incremental increase will take effect in 2004-05; the second incremental increase will take effect in 
2007-08; the third in 2010-11 and the last in 2013-14. 



Starting Points and Intermediate Goals 


Originally Set January 30, 2003 




Grades 3-8 (%) 


Grade 10 (%) 


Year 


Reading 


Mathematics 


Modified 
October 12, 
2006 


Reading 


Mathematics 


Modified 
January 5, 2005 


Starting Points 
(2001-02) 


68.9 


74.6 


52.0 


54.9 


2002-03 


68.9 


74.6 


52.0 


54.9 


2003-04 


68.9 


74.6 


52.0 


54.9 


Reading/ 

Language 

Arts 


Mathematics 


2004-05 


76.7 


81.0 


Mathematics 


— 


— 


35.4 


70.8 


2005-06 


76.7 


— 


65.8 


— 


— 


35.4 


70.8 


2006-07 


76.7 


— 


65.8 


— 


— 


35.4 


70.8 


2007-08 


84.4 


— 


77.2 


— 


— 


56.9 


80.5 


2008-09 


84.4 


— 


77.2 


— 


— 


56.9 


80.5 


2009-10 


84.4 


— 


77.2 


— 


— 


56.9 


80.5 


2010-11 


92.2 


— 


88.6 


— 


— 


78.4 


90.2 


2011-12 


92.2 


— 


88.6 


— 


— 


78.4 


90.2 


2012-13 


92.2 


— 


88.6 


— 


— 


78.4 


90.2 


2013-14 


100 


— 


100 


— 


— 


100 


100 



Note: Calculations carry fuf 



precision (see page 21) until final rounding. 
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PRINCIPLE 4. State makes annual decisions about the achievement of all public 
schools and LEAs. 



CRITICAL ELEMENT 


EXAMPLES FOR 

MEETING REQUIREMENTS 


EXAMPLES OF 
NOT MEETING 
REQUIREMENTS 


4.1 How does the State 
Accountability System 
make an annual 
determination of whether 
each public school and LEA 
in the State made AYP? 


AYP decisions for each public 
school and LEA are made 
annually. 4 


AYP decisions for public schools 
and LEAs are not made annually. 



STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 



LEAs will make preliminary determinations of AYP for schools and LEAs each year following 
the annual state testing based on data for that year. AYP decisions for public schools, LEAs and 
the state will be confirmed and reported by the state annually as part of the ABCs and NC State 
Report Card after data are collected from the LEAs. 



4 Decisions may be based upon several years of data and data may be averaged across grades within a 
public school [§1111 (b)(2)(J)]. 
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PRINCIPLE 5. All public schools and LEAs are held accountable for the 
achievement of individual subgroups. 



CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING REQUIREMENTS 


EXAMPLES OF 

NOT MEETING REQUIREMENTS 


5.1 How does the definition of 


Identifies subgroups for defining 


State does not disaggregate data 


adequate yearly progress 


adequate yearly progress: 


by each required student 


include all the required 


economically disadvantaged, 


subgroup. 


student subgroups? 


major racial and ethnic groups, 
students with disabilities, and 
students with limited English 
proficiency. 

Provides definition and data 
source of subgroups for adequate 
yearly progress. 




STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 


North Carolina master accountability files contain data for identifying the subgroups to be used 
for AYP. They are: 

• The school as a whole 

• Asian 

• American Indian 

• Black 

• Hispanic 

• Multi-racial 

• White 


• Economically disadvantaged 




• Limited English Proficient 




• Students with Disabilities 
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CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING REQUIREMENTS 


EXAMPLES OF 

NOT MEETING REQUIREMENTS 


5.2 How are public schools 
and LEAs held 
accountable for the 
progress of student 
subgroups in the 
determination of adequate 
yearly progress? 


Public schools and LEAs are held 
accountable for student subgroup 
achievement: economically 
disadvantaged, major ethnic and 
racial groups, students with 
disabilities, and limited English 
proficient students. 


State does not include student 
subgroups in its State 
Accountability System. 


STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 


Each identified subgroup must attain AYP annual measurable objectives. Some may do so using 
the confidence interval (see Critical Element 3.2), “safe harbor” or growth model provisions. 
Schools and/or subgroups that use these provisions will be appropriately noted in AYP reporting 
as having used the confidence interval, safe harbor or growth. 



CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING REQUIREMENTS 


EXAMPLES OF 

NOT MEETING REQUIREMENTS 


5.3 How are students with 
disabilities included in the 
State’s definition of 
adequate yearly progress? 


All students with disabilities 
participate in statewide 
assessments: general 
assessments with or without 
accommodations or an alternate 
assessment based on grade level 
standards for the grade in which 
students are enrolled. 

State demonstrates that students 
with disabilities are fully included 
in the State Accountability 
System. 


The State Accountability System 
or State policy excludes students 
with disabilities from participating 
in the statewide assessments. 

State cannot demonstrate that 
alternate assessments measure 
grade-level standards for the 
grade in which students are 
enrolled. 
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STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 



All students with disabilities must be assessed either with the state’s multiple choice reading 
and mathematics tests (EOGs or EOCs) administered under standard conditions, or with 
accommodations, and the state’s Grade 10 Writing Assessment administered under standard 
conditions, or with accommodations, or take one of the state’s alternate assessments as 
indicated in the student’s IEP. Results from students taking these assessments are included in 
the AYP calculations for reading/language arts and mathematics. 

The state also computes and reports the numbers and percentages of students with disabilities 
participating in the assessments. 

North Carolina has a specifically designed alternate assessment that an IEP team can designate 
for students with significant cognitive disabilities. It uses alternate achievement standards for 
these students that are aligned with North Carolina’s academic content standards and reflect 
professional judgment of the highest learning standards possible for these students. It is 
intended that the percentage of students held to these alternate achievement standards and 
deemed proficient at the district and State levels will not exceed 1.0 percent of all students in 
the grades assessed. If an LEA exceeds the 1.0 percent cap for students with the most 
significant cognitive disabilities scoring proficient on the alternate assessment the LEA may 
submit a request for an exception to the Superintendent of Public Instruction. The 
Superintendent of Public Instruction will make a decision regarding the exception and will 
report the action to the State Board of Education. For any LEAs that exceed the 1% cap and an 
exception is not granted, the state will devise a plan to randomly determine which of the 
students proficient scores will remain as proficient and the remaining scores will be changed to 
non-proficient for accountability purposes at the school, LEA and state level to attain the 1% 
cap. These changes in scores for accountability purposes will not affect what is recorded in the 
student’s records or reported to the parent(s) or guardians. 

North Carolina will implement the Secretary’s flexibility regarding modified academic 
achievement standards (refer to: www.ed.gov/policy/speced/guid/secletter/051214a.html) . 

North Carolina will include (up to a 2.0 percent cap at the district and State level consistent 
with the USED Notice of Proposed Rulemaking on December 15, 2005) students who score 
proficient on the North Carolina EXTEND2 assessment, which assesses grade level academic 
standards and is based on modified achievement standards. 

Evidence: Revised APA Policy HSP-C-021 

Starting with the 2003-04 school year, the SBE will allow a process for medical exclusions 
from testing for students with serious medical conditions. 
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CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING REQUIREMENTS 


EXAMPLES OF 

NOT MEETING REQUIREMENTS 


5.4 How are students with 


All LEP student participate in 


LEP students are not fully 


limited English proficiency 


statewide assessments: general 


included in the State 


included in the State’s 
definition of adequate 
yearly progress? 


assessments with or without 
accommodations or a native 
language version of the general 
assessment based on grade level 
standards. 

State demonstrates that LEP 
students are fully included in the 
State Accountability System. 


Accountability System. 
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STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 



All LEP students must be assessed either with the state’s multiple choice reading and 
mathematics tests administered under standard conditions, or with accommodations, and the 
Grade 10 Writing Assessment or take the state’s alternate assessment for LEP students except for 
LEP students described below. The alternate assessment is based on grade level standards and is 
not the assessment instrument used for students with the most significant cognitive disabilities. 

The state calculates and reports the numbers of LEP students participating in the state’s 
assessments. 

On February 20, 2004, US Secretary of Education, Rod Paige, issued a letter offering states some 
flexibility as it relates to testing limited English proficient (LEP) students in their first year in US 
schools and how the scores can be used for determining adequate yearly progress (AYP). 

See http://www.ed.gov/policv/gen/guid/secletter/040220.html for a copy of the letter. Based on 
this flexibility the following procedures will be followed: 

Schools shall: 

1. continue to administer state reading and mathematics tests for LEP students who score at 
or above Intermediate High on the reading section of the language proficiency test during 
their first year in US schools. Results from these assessments will be included in the 
ABCs and AYP. 

2. not require LEP students (who score below Intermediate High on the reading section of 
the language proficiency test) in their first year in US schools to be assessed on the 
reading End of Grade tests, High School Comprehensive Test in Reading, or the alternate 
assessment for reading. 

3. for purposes of determining the 95% tested rule in reading, use the language proficiency 
test from the spring administration for these students. 

4. not count mathematics results in determining AYP or ABCs performance composite 
scores for LEP students who score below Intermediate High on the reading section of the 
language proficiency test in their first year in US schools. 

5. for students previously identified as LEP, who have exited LEP identification during the 
last two years, be included in the calculations for determining the status of the LEP 
subgroup for AYP only if that subgroup already met the minimum number of 40 students 
required for a subgroup. 

Evidence: Revised APA Policy HSP-C-005 

Starting in 2004-05, LEP students who were in their first year in US schools who scored below 
Intermediate High on the reading section of the language proficiency test in the year they took 
the English I course are counted as proficient based on their performance on the Grade 10 
Writing Assessment alone. Students who are LEP and are in their first year in US schools in 
grade 10 who scored below Intermediate High on the reading section of the language proficiency 
test are not included in the calculations for proficiency at the high school level. 
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CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING REQUIREMENTS 


EXAMPLES OF 

NOT MEETING REQUIREMENTS 


5.5 What is the State's 

definition of the minimum 
number of students in a 
subgroup required for 
reporting purposes? For 
accountability purposes? 


State defines the number of 
students required in a subgroup for 
reporting and accountability 
purposes, and applies this 
definition consistently across the 
State. 5 

Definition of subgroup will result in 
data that are statistically reliable. 


State does not define the required 
number of students in a subgroup 
for reporting and accountability 
purposes. 

Definition is not applied 
consistently across the State. 

Definition does not result in data 
that are statistically reliable. 



STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 



The SBE has adopted 40 students’ scores as the minimum number of scores to be statistically reliable and 
valid for AYP puiposes. This number is based on the students that meet the definition of a full academic 
year. In the ABCs, the minimum ‘n’ = 30 requirement applies to the ABCs growth composite scores. 

Those scores are a weighted average of grade and subject level growth components, across all grades and 
subjects in the school. In the growth composites, reading and mathematics growth results arc combined, 
and one overall index is created for the entire school. The ABCs components are combined in a 
compensatory manner to produce the overall composite. There are no subgroup growth composites. 

The minimum N count used in determining the AYP status of LEAs is 40, or 1 % of the tested students, 
whichever is greater. This minimum N will be applied in each LEA to grades 3 through 8 as a group and 
high school as a group. However, in order for an LEA to enter Improvement Status, the LEA must not 
make AYP in the same subject area (reading/language aits or mathematics even if because of the 95% rule) 
or other academic indicator in each of the following grade spans [3-5, 6-8, and high school] for two 
consecutive years. 

The ABCs growth composites benefit from the averaging and weighting that takes place. The averaging 
helps reduce the volatility that would be more inherent in having multiple summary statistics. The 
weighting (each component is weighted proportional to the number of scores involved) helps reduce the 
impact of the volatility associated with smaller groups of students in some grades or subjects. The 
compensatory nature of the growth composites allows excellent growth in one grade or subject to 
compensate for a lesser growth in another subject or grade. 

In contrast, with AYP, reading/language aits and mathematics analyses arc separated and multiple summary 
statistics are created (i.e., a percentage for each subject/subgroup combination). AYP calculations are 
combined in a conjunctive manner to arrive at an AYP designation. So one subgroup performance in one 
grade cannot compensate for poor performance by another subgroup or grade in determining the school’s 
overall status. 

It seems appropriate to have a more stringent criterion for minimum ‘n’ for AYP calculations than for ABCs 
growth because: 1) AYP requires multiple summary statistics for each school to make an AYP designation 
(ABCs uses only one growth composite to make a determination of whether a school made expected 
growth); 2) AYP is a conjunctive standard (ABCs growth uses a compensatory one); and, 3) AYP 
represents even higher stakes than the ABCs. 

The agency will continue to report scores for groups of five (5) or more students. 



5 The minimum number is not required to be the same for reporting and accountability. 



30 








EXAMPLES FOR 


EXAMPLES OF 


CRITICAL ELEMENT 


MEETING REQUIREMENTS 


NOT MEETING REQUIREMENTS 


5.6 How does the State 


Definition does not reveal 


Definition reveals personally 


Accountability System 
protect the privacy of 
students when reporting 
results and when 
determining AYP? 


personally identifiable information. 6 


identifiable information. 



STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 



AYP: 

For schools in the state with less than 40 students assessed in the tested grades (other than K-2 
schools, special education schools, hospital schools and vocational and career centers where a 
feeder pattern is used to determine AYP), the North Carolina Department of Public Instruction 
(NCDPI) will use whatever state test data are available to make an AYP determination for the 
school, regardless of the numbers of students below 40, although the results for the school will be 
flagged as “results based on less than 40 students.” 

Site visits by a team or teams of educators from the School Improvement Division for schools with 
two (2) or fewer students meeting the FAY requirements will be conducted effective with the 
2003-04 school year to make AYP designations. 

Reporting: 

Aggregate results will not be reported when based on fewer than five scores. Instead, the report 
will show (or other symbol) with an appropriate explanation. 

As suggested by the USED, whenever all students in a subgroup in a school, LEA or the state score 
in the same achievement level, NC will report the results as “>95%” of the students score in the 
respective category. A consequence of complying with this privacy protection requirement may be 
that the underlying score summary information will not be completely verifiable to the schools, 
LEAs or the general public and reported summary information may not appear to aggregate 
accurately in such cases. This is an unavoidable consequence of compliance. 



6 The Family Education Rights and Privacy Act (FERPA) prohibits an LEA that receives Federal funds 
from releasing, without the prior written consent of a student’s parents, any personally identifiable 
information contained in a student’s education record. 
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PRINCIPLE 6. State definition of AYP is based primarily on the State’s academic 
assessments. 



CRITICAL ELEMENT 


EXAMPLES FOR 

MEETING REQUIREMENTS 


EXAMPLES OF 

NOT MEETING 
REQUIREMENTS 


6.1 How is the State’s 


Formula for AYP shows that 


Formula for AYP shows that 


definition of adequate 


decisions are based primarily on 


decisions are based primarily on 


yearly progress based 


assessments. 7 


non-academic indicators or 


primarily on academic 




indicators other than the State 


assessments? 


Plan clearly identifies which 
assessments are included in 
accountability. 


assessments. 



STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 



For North Carolina, AYP will be based on the End-of-Grade Reading Comprehension and 
Mathematics tests administered annually in grades 3-8; and on a combination of the English I 
End-of-Course test and the Grade 10 Writing Assessment, and the Algebra I End-of-Course test 
in grade 10. In addition the results for students who take the alternate assessment administered in 
grades 3-8 and 10 are included. For students who are following a course of study that does not 
require them to take the English I or Algebra I End-of-Course tests, or the Grade 10 Writing 
Assessment, AYP is based on the NC High School Comprehensive Test of Reading and 
Mathematics, or its alternate assessments, until other options are available. Finally, AYP also 
includes performance on the other academic indicators of attendance and graduation rates at the 
school as a whole level. 



7 State Assessment System will be reviewed by the Standards and Assessments Peer Review Team. 
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PRINCIPLE 7. State definition of AYP includes graduation rates for public High schools 
and an additional indicator selected by the State for public Middle and public Elementary 
schools (such as attendance rates). 



CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING REQUIREMENTS 


EXAMPLES OF 

NOT MEETING REQUIREMENTS 


7.1 What is the State definition 
for the public high school 
graduation rate? 


State definition of graduation rate: 

• Calculates the percentage 
of students, measured 
from the beginning of the 
school year, who graduate 
from public high school 
with a regular diploma (not 
including a GED or any 
other diploma not fully 
aligned with the state’s 
academic standards) in 
the standard number of 
years; or, 

• Uses another more 
accurate definition that 
has been approved by the 
Secretary; and 

• Must avoid counting a 
dropout as a transfer. 

Graduation rate is included (in the 
aggregate) for AYP, and 
disaggregated (as necessary) for 
use when applying the exception 
clause 8 to make AYP. 


State definition of public high 
school graduation rate does not 
meet these criteria. 



See USC 631 l(b)(2)(I)(i), and 34 C.F.R. 200.20(b) 
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STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 



The state proposes to start with a “simple” definition of graduation rates based on the actual definition 
contained in the NCLB legislation. The state acknowledges the new definition contained in the final 
regulations and will migrate to a more desirable cohort definition over the next four years. The current 
plan is as follows: 

“Simple” Definition 

To be within DPI implementation capabilities for 2002-03, the proposed definition would be based on the 
wording provided in the NCLB legislation. 

In that framework, we propose that the North Carolina “graduation rate for public high schools” be 
initially defined as “The percentage of students who graduate from high school with a regular diploma in 
the standard number of years.” According to final regulations, “diploma” does not include “an alternative 
degree that is not fully aligned with the State’s academic standards, such as a certificate or a GED.” SBE 
Policy HSP-N-004 describes the requirements for a North Carolina diploma and provides evidence that 
North Carolina issues only one diploma to all students, regardless of which Course of Study they 
successfully complete. See http://sbepolicy.dpi.state.nc.us/ for a copy of this policy. The “standard 
number of years” will be defined as four years or less. 

Data source for 2002-03 : The 2002-03 ABCs masterbuild files will be coded to reflect diploma 
recipients. The same files will indicate the date when students took End-of-Grade (EOG) tests in 8 lh 
grade. Calculating the elapsed time between 8th grade EOG tests and diploma receipt will ascertain the 
number of years. Schools will be given the capability to manually record the necessary information for 
students that do not have 8 th grade EOG data (e.g., students who moved into the state after the 8 th grade or 
who previously attended private schools). 

Timeline for Moving to a Cohort Definition 

of Graduation Rate 

For the longer term, we propose a prospective (forward) on-time graduation rate. The first step would be 
to establish a baseline for membership in ninth grade during the 2002-03 school year. Current student 
information management systems in North Carolina do not have the capability to track students over a 
four-year period anywhere in the state. Therefore, LEAs would be asked to generate student rosters for 
ninth graders in 2002-03 and retain them for future reference. LEAs and/or schools would record the 
transition outcomes for each student on the roster over the next four years and maintain that information 
so that it could be matched with the diploma recipient information collected through the 2005-06 
masterbuild files (or other data collection that may be more appropriate at that time). Thus the first year 
in which a cohort based graduation rate could be calculated would be the 2005-06 school year, and the 
first year in which “progress” could be ascertained using a cohort definition for two successive 
graduation rates would be 2006-07. 

The calculations will avoid counting a dropout as a transfer. The graduation rate will be used for AYP 
for the school as a whole and to invoke the exception clause (“safe harbor”) as appropriate for 
determining AYP for groups in a school. The graduation rate will be the other academic indicator for 
schools that have a twelfth grade and graduate seniors. Progress will be defined as at least 0.1 percentage 
point increase from one year to the next (up to a threshold of 90%). Any fluctuations above 90% for the 
graduation rate will meet the requirement for progress. 
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CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING REQUIREMENTS 


EXAMPLES OF 

NOT MEETING REQUIREMENTS 


7.2 What is the State’s 
additional academic 
indicator for public 
elementary schools for the 
definition of AYP? For 
public middle schools for 
the definition of AYP? 


State defines the additional 
academic indicators, e.g., 
additional State or locally 
administered assessments not 
included in the State assessment 
system, grade-to-grade retention 
rates or attendance rates. 9 

An additional academic indicator 
is included (in the aggregate) for 
AYP, and disaggregated (as 
necessary) for use when applying 
the exception clause to make 
AYP. 


State has not defined an 
additional academic indicator for 
elementary and middle schools. 


STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 


Attendance rates will be used for elementary and middle schools for the other academic 
indicator. It will be included (in the aggregate) to determine AYP and will be disaggregated (as 
necessary) for applying the exception clause (“safe harbor”) to groups within the school. 

Progress on attendance will be defined as at least 0. 1 percentage point increase from one year to 
the next up to a threshold of 90%. Any fluctuations above 90% for attendance will meet the 
requirement for progress. 



9 NCLB only lists these indicators as examples. 
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CRITICAL ELEMENT 



EXAMPLES FOR 
MEETING REQUIREMENTS 



EXAMPLES OF 

NOT MEETING REQUIREMENTS 



7.3 Are the State’s academic State has defined academic State has an academic indicator 

indicators valid and indicators that are valid and that is not valid and reliable, 

reliable? reliable. 

State has an academic indicator 
State has defined academic that is not consistent with 

indicators that are consistent with nationally recognized standards, 
nationally recognized standards, if 

any. State has an academic indicator 

that is not consistent within grade 
levels. 



STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 

Yes, they were required or recommended by the NCLB Act of 2001. 
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PRINCIPLE 8. AYP is based on reading/language arts and mathematics 
achievement objectives. 



CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING REQUIREMENTS 


EXAMPLES OF 
NOT MEETING 
REQUIREMENTS 


8.1 Does the state measure 
achievement in 
reading/language arts and 
mathematics separately for 
determining AYP? 


State AYP determination for 
student subgroups, public 
schools and LEAs separately 
measures reading/language arts 
and mathematics. 10 

AYP is a separate calculation for 
reading/language arts and 

mathematics for each group, 
public school, and LEA. 


State AYP determination for 
student subgroups, public 
schools and LEAs averages or 
combines achievement across 
reading/language arts and 
mathematics. 



STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 



Yes. 



10 If the state has more than one assessment to cover its language arts standards, the State must create 
a method for including scores from all the relevant assessments. 
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PRINCIPLE 9. State Accountability System is statistically valid and reliable. 



CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING REQUIREMENTS 


EXAMPLES OF 
NOT MEETING 
REQUIREMENTS 


9.1 HowdoAYP 

determinations meet the 
State’s standard for 
acceptable reliability? 


State has defined a method for 
determining an acceptable level of 
reliability (decision consistency) 
for AYP decisions. 

State provides evidence that 
decision consistency is (1) within 
the range deemed acceptable to 
the State, and (2) meets 
professional standards and 
practice. 

State publicly reports the estimate 
of decision consistency, and 
incorporates it appropriately into 
accountability decisions. 

State updates analysis and 
reporting of decision consistency 
at appropriate intervals. 


State does not have an 
acceptable method for 
determining reliability (decision 
consistency) of accountability 
decisions, e.g., it reports only 
reliability coefficients for its 
assessments. 

State has parameters for 
acceptable reliability; however, 
the actual reliability (decision 
consistency) falls outside those 
parameters. 

State’s evidence regarding 
accountability reliability (decision 
consistency) is not updated. 
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STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 



The state will monitor the AYP outcomes yearly and examine their characteristics, including 
consistency across years and in relation to other indicators of progress. The Compliance 
Commission for Accountability and the Division of Accountability Services Technical Advisory 
Committee will be consulted for advice regarding how “decision consistency” can be defined in 
the context of AYP and NCLB. 

North Carolina has a history of giving great attention to the reliability and validity of its 
assessments and accountability system. All assessments are developed to meet or exceed 
industry standards for psychometric validity and reliability. The ABCs system was originally 
developed empirically using replication and cross-validation procedures. For example, the 
growth standards were based on data for three years (two successive, independent, two-wave 
longitudinal cohorts.) Similarly, End of Course (EOC) prediction formulas (for high school 
growth standards) were checked with cross-validation samples during the developmental years 
and by replication of the models in subsequent years. All models for setting growth standards 
have been monitored annually to verify that they continue to operate according to their 
respective designs, in an equitable and consistent manner. In addition, each year all data 
collection and analysis procedures are subject to numerous quality control checks and are 
subjected to verification at the school, LEA, and state levels. 

The NCDPI will adopt a similar attitude with respect to the system for calculating AYP. The 
system and the results will be monitored annually. The processes for data collection and 
analysis will be monitored to ensure that they continue to comply with existing quality standards. 
Additional quality control standards will be developed as necessary to address specific 
requirements related to NCLB. Verification will be required at the school, LEA and state levels. 

The validity and reliability of the system will be studied and monitored. The NCDPI plans to 
consider and use, whenever appropriate, ideas suggested in the papers referenced in the Peer 
Review Guidance document issued by USED — specifically, the references: 

- Marion, S.F., White, C., Carlson, D., Erpenbach, W.J., Rabinowitz, S, Sheinker, J. (2002). 
Making valid and reliable decisions in the determination of adequate yearly progress: A Paper in the 
Series: Implementing The State Accountability System Requirements Under The No Child Left Behind 
Act of 2001. Washington, D.C.: Council of Chief State School Officers. 

- Hill, R.. & DePascale, C. (2002). Determining the reliability of school scores. National Center 
for the Improvement of Educational Assessment. 

- Baker, E. & Linn, R.L. (2002). Validity issues for accountability systems. CRESST/CSE 
Technical Report #585. 

Baker, E., Linn, R.L., Herman, J., & Koretz, D. (2002, Winter). Standards for 
educational accountability systems. CRESST Policy Brief 5. 

Validity and reliability of accountability systems is a fairly new concept, as evidenced by the 
uniformly recent publication dates of the suggested papers. NCDPI will monitor the literature in 
this area and refine its efforts in this regard as the field evolves over the next few years to 
address this important concern. 
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CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING REQUIREMENTS 


EXAMPLES OF 
NOT MEETING 
REQUIREMENTS 


9.2 What is the State's process 
for making valid AYP 
determinations? 


State has established a process 
for public schools and LEAs to 
appeal an accountability decision. 


State does not have a system for 
handling appeals of accountability 
decisions. 



STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 



The state already has an appeals process related to growth standards. The Compliance 
Commission for Accountability serves as the appeals committee and advises the SBE in matters 
related to appeals. However, for AYP determinations, the LEAs will be required to establish an 
appeals process for their respective schools. 

Before identifying a school for school improvement, corrective action, or restructuring, an LEA 
must provide the school with an opportunity to review the school-level data, including academic 
assessment data, on which the proposed identification is based. 

The SBE has designated the Title I Committee of Practitioners as the review body for an LEA 
appealing its being considered for LEA Improvement Status. The LEA will have two weeks to 
appeal and the SBE authorizes the State Superintendent to make a final determination for the 
LEA. 
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EXAMPLES FOR 

CRITICAL ELEMENT MEETING REQUIREMENTS 



State has a plan to maintain 
continuity in AYP decisions 
necessary for validity through 
planned assessment changes, 
and other changes necessary to 
comply fully with NCLB. 11 

State has a plan for including new 
public schools in the State 
Accountability System. 

State has a plan for periodically 
reviewing its State Accountability 
System, so that unforeseen 
changes can be quickly 
addressed. 



STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 

The state transitioned to a new mathematics assessment two years ago and will experience a 
change in Reading Comprehension tests in 2002-03. Each year the Division of Accountability 
Services does analyses to monitor the stability of the current accountability model. These 
analyses will continue under the new system. Equating, linking and comparability studies are 
used as necessary to ensure that transitions can be made appropriately. These issues will be 
discussed with the state’s Technical Advisory Committee. We also are planning for additional 
components (e.g., science) to be added to our program as required by NCLB. 



11 Several events may occur which necessitate such a plan. For example, (1) the State may need to 
include additional assessments in grades 3-8 by 2005-2006; (2) the State may revise content and/or 
academic achievement standards; (3) the State may need to recalculate the starting point with the 
addition of new assessments; or (4) the State may need to incorporate the graduation rate or other 
indicators into its State Accountability System. These events may require new calculations of validity and 
reliability. 



9.3 How has the State planned 
for incorporating into its 
definition of AYP 
anticipated changes in 
assessments? 



EXAMPLES OF 
NOT MEETING 
REQUIREMENTS 

State’s transition plan interrupts 
annual determination of AYP. 

State does not have a plan for 
handling changes: e.g., to its 
assessment system, or the 
addition of new public schools. 
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PRINCIPLE 10. In order for a public school or LEA to make AYP, the State 
ensures that it assessed at least 95% of the students enrolled in each subgroup. 



CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING REQUIREMENTS 


EXAMPLES OF 
NOT MEETING 
REQUIREMENTS 


1 0.1 What is the State's method 
for calculating participation 
rates in the State 
assessments for use in 
AYP determinations? 


State has a procedure to 
determine the number of absent 
or untested students (by 
subgroup and aggregate). 

State has a procedure to 
determine the denominator (total 
enrollment) for the 95% 
calculation (by subgroup and 
aggregate). 


The state does not have a 
procedure for determining the 
rate of students participating in 
statewide assessments. 

Public schools and LEAs are not 
held accountable for testing at 
least 95% of their students. 




Public schools and LEAs are held 
accountable for reaching the 95% 
assessed goal. 





STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 



Membership rosters will be collected from all schools. We have implemented testing 
requirements for several years in which we determine violations of percent tested rules. The 
accountability system may deny incentive awards if schools do not test the required percentage. 
Data are crosschecked with other membership reports to ascertain accuracy. Students with 
invalid assessment scores are counted as non-participants. Participation rate will be calculated 
using the number of students with valid assessment scores divided by the number of students in 
membership on the first day of spring testing. 
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EXAMPLES FOR 


EXAMPLES OF 


CRITICAL ELEMENT 


MEETING REQUIREMENTS 


NOT MEETING 
REQUIREMENTS 


10.2 What is the State's policy 


State has a policy that 


State does not have a procedure 


for determining when the 


implements the regulation 


for making this determination. 


95% assessed 


regarding the use of 95% 




requirement should be 


allowance when the group is 




applied? 


statistically significant according 
to State rules. 





STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 



The 95% rule will apply whenever the number of students in membership in a group is at least 
40. 
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Appendix A 

Required Data Elements for State Report Card 

1111(h)(1)(C) 

1 . Information, in the aggregate, on student achievement at each proficiency level on the State 
academic assessments (disaggregated by race, ethnicity, gender, disability status, migrant status, 
English proficiency, and status as economically disadvantaged, except that such disaggregation 
shall not be required in a case in which the number of students in a category is insufficient to 
yield statistically reliable information or the results would reveal personally identifiable 
information about an individual student. 

2. Information that provides a comparison between the actual achievement levels of each student 
subgroup and the State’s annual measurable objectives for each such group of students on each 
of the academic assessments. 

3. The percentage of students not tested (disaggregated by the student subgroups), except that 
such disaggregation shall not be required in a case in which the number of students in a category 
is insufficient to yield statistically reliable information or the results would reveal personally 
identifiable information about an individual student. 

4. The most recent 2-year trend in student achievement in each subject area, and for each grade 
level, for the required assessments. 

5. Aggregate information on any other indicators used by the State to determine the adequate 
yearly progress of students in achieving State academic achievement standards disaggregated by 
student subgroups. 

6. Graduation rates for secondary school students disaggregated by student subgroups. 

7. Information on the performance of local educational agencies in the State regarding making 
adequate yearly progress, including the number and names of each school identified for school 
improvement under section 1116. 

8. The professional qualifications of teachers in the State, the percentage of such teachers 
teaching with emergency or provisional credentials, and the percentage of classes in the State not 
taught by highly qualified teachers, in the aggregate and disaggregated by high-poverty 
compared to low-poverty schools which (for this purpose) means schools in the top quartile of 
poverty and the bottom quartile of poverty in the State. 
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Appendix B 



Chronology of Amendments to the Accountability Workbook 
Changes to AYP rules implemented in 2003-04 

The State Board of Education and the U.S. Department of Education (USED) approved changes 
in the process for determining AYP for the 2003-04 accountability year that included: 

1. Adding a new ABCs recognition category to denote schools that made expected or 
high growth and met AYP (January 08, 2004). 

2. Naming the new category Honor Schools of Excellence (March 04, 2004). 

3. Allowing a process for medical exclusions from testing for students with serious 
medical conditions (March 04, 2004). 

4. Averaging participation rates for the last two or three years, depending on how many 
years of data were available, whenever a school did not meet the 95% participation 
standard for the current year (May 06, 2004). 

5. Providing for special analysis of AYP for targeted assistance schools (TAS) (March 
04, 2004). 

6. Making these additional TAS analyses contingent upon the LEA “tagging” the 
appropriate students for the analyses (May 06, 2004). 

7. Applying a 95% confidence interval to the percentages of students scoring proficient 
in reading and/or mathematics to determine if subgroups met the AMO (January 08, 
2004). 

8. Requiring a Title I school not making AYP to miss targets in the same subject (e.g., 
reading) for two successive years before entering Title I School Improvement (January 
08, 2004). 

9. Requiring non-Title I schools not making AYP to miss targets in same subject area 
(March 04, 2004). 

10. Placing a 1.0 percent cap on the percentage of students with the most significant 
cognitive disabilities held to alternate achievement standards and deemed proficient 
(through the Alternate NCAAP, and NCAAAI administered at least three years or 
more below students’ assigned grade levels) at the district and State levels (March 04, 
2004). 

11. Not requiring LEP students (who score below Intermediate High on the reading 
section of the language proficiency test) in their first year in U.S. schools to be 
assessed on the reading End-of-Grade tests, High School Comprehensive Test in 
Reading, or the NC Alternate Assessment Academic Inventory (NCAAAI) for 
reading. (Schools were to use the language proficiency test for determining the 95% 
participation; mathematics results were not used in determining AYP or ABCs 
performance composite scores for LEP students who scored below Intermediate High.) 
Students previously identified as LEP who exited LEP identification during the last 
two years were included in the AYP calculations only if the subgroup already met the 
minimum number of 40 students (March 04, 2004). 

12. Setting the minimum N count used in determining the AYP status of LEAs at 40 or 
1% of the full academic year (FAY) students, whichever was greater (July 1, 2004). 

13. Providing that an LEA could enter Improvement Status only if the LEA did not make 
AYP in the same subject area in both the 3-8 and high school grade spans for two 
consecutive years (July 1, 2004). 
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14. Allowing site visits by a School Improvement team to make AYP designations for 
schools with two (2) or fewer students meeting the FAY requirements (July 1, 2004). 

Changes implemented in 2004-05 

The State Board of Education and the U.S. Department of Education (USED) approved changes 

in the process for determining AYP for the 2004-05 accountability year that included: 

15. Defining Full Academic Year (FAY) for Grade 10 as 140 days in membership as of 
the first day of spring EOC testing (February 03, 2005). 

16. Using Algebra I and a combination of English I and the Grade 10 Writing Assessment 
to determine AYP in mathematics and language arts at grade 10. New starting points 
for Grade 10 were set as a result: those are 35.4% proficiency in reading/language arts, 
and 70.8% in mathematics (February 03, 2005). 

17. Counting LEP students who were in their first year in US schools who scored below 
Intermediate High on the reading section of the language proficiency test in the year 
they took the English I course as proficient based on their performance on the Grade 
10 Writing Assessment alone. Students who are LEP and are in their first year in US 
schools in grade 10 who score below Intermediate High on the reading section of the 
language proficiency test are not included in the calculations for proficiency at the 
high school level (February 03, 2005). 

18. Providing that an LEA could enter Improvement Status only if the LEA did not make 
AYP in the same subject area in each of three grade spans: elementary, middle, and 
high school, for two consecutive years (March 02, 2005). 

19. Allowing the LEA two weeks to appeal its LEA Improvement Status and authorizing 
the State Superintendent to make a final determination for the LEA (March 02, 2005). 

20. Placing a 2.0 percent cap on the percentage of students with persistent academic 
disabilities deemed proficient on alternate assessments (May 04, 2005). [Note: not 
approved by the USED in May, 2005.] 

21. Allowing schools that would have missed AYP solely due to the students with 
disabilities (SWD) subgroup to take advantage of the Secretary of Education’s May 
10, 2005 guidelines to use Option #1 for the 2004-05 school year. For NC it adds 
14.4% to the actual percentage of students scoring proficient in the SWD subgroup 
and this higher total percentage, without confidence interval, is compared with the 
annual measurable objective to see if the school now meets the AYP target (June 30, 

2005) . 

22. Adding that LEA and state AYP both attendance and graduation rate will be used as 
the Other Academic Indicators (OAI) and removing of references to NCAAAI and 
NCAAP alternate assessment tools due to changes in alternate assessments (November 
3, 2005). 

Changes implemented in 2005-06 

23. Describing staggered release of AYP (July 12, 2006). 

24. Including EXTEND2 students’ scores for ABCs/A YP purposes (July 12, 2006). 

25. Removing references to Option #1 for students with disabilities (July 12, 2006). 

26. Clarifying definition of participation rate (July 12, 2006). 

27. Including Other Academic Indicator for LEA Improvement determinations (July 12, 

2006) . 
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28. Including a growth model component as part of the process for determining AYP (July 
12,2006) 

29. Approving higher standards for achievement levels in mathematics for grades 3-8 and 
revising the annual measurable objectives in light of the more rigorous student 
proficiency standards (October 12, 2006). 
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